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Introduction 


Access  to  the  dataset  gives  us  a  large  enough 
record  of  traffic  to  test  hypotheses  in  network 
security. 

Given  this,  we  select  and  evaluate  various 
security  measures  against  real  traffic 

*  Or  a  reasonable  facsimile  thereof 

One  example:  target  resident  DDoS  Filters 

*  Heavily  constrain  the  problem-  not  considering  SYN 
floods,  smurfing,  reflection  attacks... 
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How  Do  We  Test? 

Any  analysis  opens  a  can  of  worms... err, 
“assumptions” 

*  The  network  constantly  changes 

*  What  is  a  representative  host? 

Rerunning  attacks  is  of  debatable  value 

*  Most  of  the  legitimate  traffic  is  dropped,  that’s  what 
a  DoS  is  for 

We  want  our  results  to  be  representative 
*Test  and  summarize  over  multiple  machines 

We  want  our  results  to  be  reproducible 

*  Depend  heavily  on  SiLK  structures  and  tools 
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Evaluation 


Trained  filters  on  15  days  of  legitimate  traffic 

*  Built  a  representation  of  IP  address:  volume 
relationship  (via  rwaddrcount) 

Then  generated  a  simulated  DoS 

*  Botnet  IPs  collected  with  rwset 

*  Normal  traffic  selected  from  another  day 

Resulting  traffic  was  then  evaluated  for  failure 
rates 

Tested  2  types  of  filters: 

*  Clustering  -  groups  of  adjacent  IP  addresses 

*  PI  -  path  marking  approach 

©  2003  by  Carnegie  Mellon  University 


False  Negative  Percentage 


CiiriM'o'ie  MHIon 

CERT 

~  Software  Engineering  Institute 

Analysis 

Center 

DoS  Filters 


©  2003  by  Carnegie  Mellon  University  6 


Carnegie  Mel  Ion 


Software  Engineering  Institute 


Initial  Observations 


Analysis 

Center 


Two  groups 

*  One  group  assumes  a  magic  DoS  Detection  Oracle 

•  That’s  the  group  with  better  results 

In  general,  the  filters  don’t  do  well 

*  Should  we  compare  IP  addresses,  or  packets? 

*  Is  traffic  different  for  different  servers? 

Let’s  look  at  one  result  in  more  depth 
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One  result  in  more  depth 


Comparative  Failure  Rates  For  90%  threshold,  25  Days  Leambig  Time 
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Observations 


Normal  traffic  varies  extensively 

*  Although  it  seems  to  vary  more  with  “smaller” 
servers 

*  And  it’s  better  when  you  look  at  packet  counts 

•  Which  makes  sense,  given  the  absurd  number  of 
scanners  we  see. 

False  negative  rate  (attackers  accepted)  seems  to 
be  related  to  server  activity  -  the  busier  the 
higher. 

*  Attackers  don’t  vary  as  much 
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In  the  majority  of  cases,  packets  are  dropped 

because  they’ve  never  been  seen  before 

*  Short  learning  curves  -  effectively  no  change  in 
false  positive  rate  after  a  week  of  learning. 

*  Especially  true  for  spoofed  traffic 

Entropy  is  lower  than  expected 

*  Filters  that  rely  on  spoof  defense  (HOF,  PI)  drop  less 
than  10%  of  their  packets  because  they  detect  a 
spoof 
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Further  Work 


Exploiting  our  DoS  attack  traffic  records  further 

*  We  know  how  the  network  reacts 

*  We  know  how  the  attack  starts  and  ends 

•  Which  impacts  learning  curve  for  defenses  that  only 
profile  the  attack 

Further  use  of  other  network  maps 

*  Skitter  (used  for  PI),  &c. 

Formalization  of  the  techniques  used 

*  Developed  a  matrix  based  approach  for  the  final 
iteration 

*  Tools  are  going  to  be  available  publicly 
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A  Final  Note 

URL  for  the  SiLK  tools: 
http://silktools.sourceforge.net 
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